1. Field of the Invention
The present invention is related to anti-malware technology, and more particularly, to detection of malware based on heuristic analyses of computer programs and application of dynamic nested behavioral rules.
2. Description of the Related Art
Detection of viruses and malware has been a concern throughout the era of the personal computer. With the growth of communication networks such as the Internet and increasing interchange of data, including the rapid growth in the use of e-mail for communications, the infection of computers through communications or file exchanges is an increasingly significant consideration. Infections take various forms, but are typically related to computer viruses, Trojan programs, rootkits or other forms of malicious code (i.e., malware).
Recent incidents of e-mail mediated virus attacks have been dramatic both for the speed of propagation and for the extent of damage, with Internet service providers (ISPs) and companies suffering service problems and a loss of e-mail capability. In many instances, attempts to adequately prevent file exchange or e-mail mediated infections significantly inconvenience computer users. Improved strategies for detecting and dealing with virus attacks are desired.
One conventional approach to detecting viruses is signature scanning. Signature scanning systems use sample code patterns extracted from known malware code and scan for the occurrence of these patterns in other program code. A primary limitation of the signature scanning method is that only known malicious code is detected, that is, only code that matches the stored sample signatures of known malicious code is identified as being infected. All viruses or malicious code not previously identified, and all viruses or malicious code created after the last update to the signature database, will not be detected.
In addition, the signature analysis technique fails to identify the presence of a virus if the signature is not aligned in the code in the expected fashion. Alternatively, the authors of a virus may obscure the identity of the virus by opcode substitution or by inserting dummy or random code into virus functions. Nonsense code can be inserted that alters the signature of the virus to a sufficient extent as to be undetectable by a signature scanning program, without diminishing the ability of the virus to propagate and deliver its payload.
Another virus detection strategy is integrity checking. Integrity checking systems extract a code sample from known, benign application program code. The code sample is stored, together with information from the program file, such as the executable program header and the file length, as well as the date and the time stamp of the sample. The program file is checked at regular intervals against this database to ensure that the program file has not been modified.
Integrity checking programs generate long lists of modified files when a user upgrades the operating system of the computer or installs or upgrades application software. A main disadvantage of an integrity check-based virus detection system is that a great many warnings of virus activity issue whenever any modification of an application program is performed. It is difficult for a user to determine when a warning represents a legitimate attack on the computer system.
One of the most effective techniques for detecting computer malware is heuristic analysis of computer programs. The heuristic analysis is a behavior-based technique in which actions of a suspect computer program are analyzed for known malicious actions, such as replication, file overwrites, attempts to hide files, attempts of registry access, sending sensitive information or receiving malicious code over a network, etc.
If one or more of the malicious actions are detected, the program is flagged as a potential malware. While heuristic analysis enabled detection of various classes of malware, heuristic analysis approach may fail to detect some forms of malware with complex behavioral patterns. In particular, heuristic analysis fail against malware performing a sequence of actions distributed between different program components or numerous applications. It is also ineffective in protecting against rootkit type of malware that acts over a long period of time. Additionally, conventional systems using heuristic analysis employ hundreds of rules. This imposes a burden on system resources, as all of the rules need to be loaded and processed by the system.
It is apparent that improved method and techniques for protecting against malware with complex behavior patterns are required. Accordingly, there is a need in the art for a system and method that addresses the need for protection against malware by detecting complex behavior patterns without employing a large number of rules.